Field
Fields are discovered automatically via Init() and then configured before Load(). The engine is schema-less — you never declare fields up front.
engine.Init(fstream);
engine.GetField("name")!.Searchable = true;
engine.GetField("name")!.Weight = 2.0f;
engine.GetField("description")!.Searchable = true;
engine.GetField("category")!.Filterable = true;
engine.GetField("category")!.Facetable = true;
engine.GetField("price")!.Filterable = true;
engine.GetField("price")!.Sortable = true;Configuration Properties
| Property | Type | Default | Description |
|---|---|---|---|
Searchable | bool | false | Include this field in the text search index |
Filterable | bool | false | Allow this field to be used in filters and boosts. Refused with an ArgumentException on a field with no type |
Facetable | bool | false | Return value/count pairs for this field with search results |
Sortable | bool | false | Allow sorting by this field. Score is always the primary sort key. Refused with an ArgumentException on a field with no type |
WordIndexing | bool | false | Build a full word index for exact single-word matching. Useful for large datasets with many repeated values |
HighResolution | bool | false | Also index N-grams over the field text with all delimiters removed (e.g. "hvis feks" additionally indexes "hvisfeks"), and strip delimiters from the query too, so a run-together or split query matches across removed delimiters. Only meaningful when Searchable |
Embeddable | bool | false | Mark this field for embedding-based (vector) search. Embeddable fields are excluded from the text search index |
PreloadFilters | bool | false | Cache a filter for each distinct value during Load. Accessible via SearchEngine.PreLoadedFilters. Faster repeated filtering at the cost of memory |
HighResolution | bool | false | Add extra delimiter-removed N-grams to sharpen run-together / split-word matches. Coverage already matches these, so this lifts precision at the cost of index space — best on a few key fields like titles. Only when Searchable |
Weight | float | 1.0f | Relative score weight for this field. Higher values make matches in this field rank higher |
BM25b | float | 0.75f | Length normalisation parameter. 0 = no normalisation, 1 = full normalisation. Applies in both BM25F and BM25PerField modes |
BM25k1 | float | 1.2f | Term-frequency saturation. Typical range 1.0–2.0. Setting different values across fields forces BM25PerField mode; uniform values select BM25F mode |
Read-Only Properties
| Property | Type | Description |
|---|---|---|
Name | string | Field name as it appears in the JSON data |
Type | JsonValueKind | Value kind: String, Number, True/False, Array, Object |
IsArray | bool | Whether this field holds an array of values |
Optional | bool | true if the field is absent or null in at least one document |
EmbeddingDimensions | int | Detected vector dimension. 0 until data is loaded with Embeddable = true |
Setting
PreloadFilters = trueautomatically setsFilterable = trueon the same field.
Fields With No Type
A field that was null or absent in every document Init() read has no type. Filterable and
Sortable are the only two roles that need one, to choose between a numeric and a categorical
index and to pick a comparator, so both are refused on such a field: TrySetFieldConfiguration returns
false with the reason, and the setters on Field throw ArgumentException.
Clearing either is always allowed.
Searchable and Facetable do not read the type and stay available. Setting Searchable on an
empty field prepares it: a record inserted later that does carry a value is indexed and found.
Changing Configuration After Load
A change needs one of three things, and you can ask which before applying it:
var proposed = new[] { new FieldProxy { FieldName = "category", Facetable = true } };
if (engine.DocumentFields.RequiresReload(proposed))
{
// Searchable, WordIndexing, Embeddable, PreloadFilters, BM25k1 and HighResolution.
// Only Load consumes these. Applied in place they return the engine to Created,
// so searches fail with a reason instead of answering as if the field were empty.
}
else if (engine.DocumentFields.RequiresReindex(proposed))
{
engine.TrySetFieldConfiguration(proposed, out string error);
engine.Index();
}
else
{
engine.TrySetFieldConfiguration(proposed, out string error); // takes effect at query time
}RequiresReindex covers a field getting its first role. A document keeps values only for the
fields in use when it was loaded, so such a field has none until Index() reads them back from the
stored JSON. A second role on a field that already had one takes effect at query time.
To apply a reload-only change without taking the engine out of service, build the new index beside
the live one with CreateInMemoryClone(monitor, fieldOverrides) and swap it in.
FieldProxy
FieldProxy is used by GetFieldConfiguration() (returns FieldProxy[]) and TrySetFieldConfiguration(FieldProxy[], out string error). All boolean and float properties are nullable — null on a set call means "leave unchanged"; every property is populated on a get call.
// Read the full configuration
FieldProxy[] config = engine.GetFieldConfiguration();
// Apply changes — only non-null values are written
engine.TrySetFieldConfiguration([
new FieldProxy { FieldName = "title", Searchable = true, Weight = 2.0f },
new FieldProxy { FieldName = "description", Searchable = true, Weight = 1.0f },
new FieldProxy { FieldName = "category", Filterable = true, Facetable = true },
new FieldProxy { FieldName = "price", Filterable = true, Sortable = true },
], out string error);| Property | Type | Description |
|---|---|---|
FieldName | string | Must match a field name discovered by Init() |
FieldType | string? | Read-only on GET ( "String", "Number", "Boolean"). Ignored on SET |
IsArray | bool? | Read-only on GET. Ignored on SET |
Searchable | bool? | Include in text search index |
Filterable | bool? | Allow use in filters and boosts |
Facetable | bool? | Return value/count pairs with results |
Sortable | bool? | Allow sorting by this field |
WordIndexing | bool? | Exact word matching in coverage |
Embeddable | bool? | Vector embedding field |
PreloadFilters | bool? | Cache a filter per distinct value during Load. Also forces Filterable = true |
HighResolution | bool? | High-resolution indexing for run-together/split-word precision. Best on a few key fields like titles |
Weight | float? | Score weight multiplier |
BM25b | float? | Length normalisation parameter |
BM25k1 | float? | TF saturation parameter |
Scoring Modes
Setting BM25k1 differently across fields controls which scoring algorithm the engine uses after Index():
// BM25F mode (default) — uniform k1, single shared index
engine.GetField("title")!.BM25k1 = 1.2f;
engine.GetField("body")!.BM25k1 = 1.2f; // same → BM25F
// BM25PerField mode — divergent k1, one index per field
engine.GetField("title")!.BM25k1 = 2.0f;
engine.GetField("body")!.BM25k1 = 1.2f; // different → BM25PerFieldInspect SearchEngine.ScoringMode after Index() to confirm which path was selected.
Field Weights
Weight is a plain float. The practical range is 0.5–3.0. The ratio between fields matters — 2.0 and 1.0 produces the same relative effect as 4.0 and 2.0.
engine.GetField("title")!.Weight = 3.0f;
engine.GetField("heading")!.Weight = 2.0f;
engine.GetField("metadata")!.Weight = 1.5f;
engine.GetField("body")!.Weight = 1.0f;There is no hard maximum, but values above 5× the lowest weight in your set offer diminishing returns. If one field needs to dominate that strongly, it is usually better to make it the only searchable field.
Constructors
Field()Parameterless constructor for deserialization.
Field(string name, JsonValueKind type, object value, bool isArray)stringnameJsonValueKindtypeobjectvalueboolisArray
DocumentFields()Default ctor.
Properties
| Name | Type | Access | Description |
|---|---|---|---|
BM25b | float | get/set | Per-field length-normalization parameter b for BM25 / BM25F scoring.
Range [0, 1]: 0 = no length normalization, 1 = full normalization.
Typical values: 0.3 for short fields (title), 0.75 for longer fields (body).
Defaults to 0.75. Applies in both(per-field BM25) and(multi-field BM25F). |
BM25k1 | float | get/set | Per-field term-frequency saturation parameter k1 for BM25 scoring.
Typical range: 1.0–2.0. Lower values saturate TF faster (diminishing
returns on repeated terms). Defaults to 1.2. |
HighResolution | bool | get/set | High-resolution indexing for this field. When true, the field text is ALSO indexed
with all delimiters removed (e.g. "hvis feks" additionally indexes
"hvisfeks"), and a search whose words are run together (or split) still matches across the
removed delimiters. The query side strips delimiters too whenever at least one searchable
field is high-resolution. Delimiters are the ones defined by the engine's configuration.
Off by default; opt in per field. |
Embeddable | bool | get/set | When true, this field holds a vector embedding — a fixed-length array of
float values produced by an embedding model. Fields must be set embeddable
after Init during the fieldconfiguration.
Embeddable fields are excluded from the standard text search index.
During Load their float[] values are collected into a per-field list that
is passed to the HNSW index for approximate nearest-neighbour search.
The expected dimensionality is fixed on the first document loaded and
validated on all subsequent ones.
Setting this to false on a field whose name matches the auto-detection
pattern overrides the automatic classification. |
EmbeddingDimensions | int | get/set | The expected number of dimensions for this embedding field.
Set automatically from the first document loaded during Load.
Zero means not yet determined. Ignored when Embeddable is false. |
Facetable | bool | get/set | Set to true if this field is going to be used as a facetable field. |
Filterable | bool | get/set | Set to true if this field is going to be used as a filterable field.
Refused on a field with no type (null in every analyzed document). |
Searchable | bool | get/set | sets whether the content should be searchable. |
Sortable | bool | get/set | Must be set in order to sort on this field.
This will be a secondary search key – score is always first.
Both this field and the sort order must be passed to Query. |
WordIndexing | bool | get/set | Enables exact word matching for single-word queries.
When enabled, records containing an exact match of the query word are prioritized in the result list.
For example, searching for "computer" will rank records containing the word "computer" topmost.
This feature is only applicable in Coverage mode. Any field marked with this property set to true
will get its values indexed for exact matching.
The shortest word is TokenizerSetup.MinimumWordSize (2 by default).
These matches are never truncated and the max word size is 8 characters.
Case sensitivity is determined by configuration settings. For array fields one matching element is considered a hit.
Exact word matches rank above other matches; this property itself does not apply any boost.
Words are split using the delimiters defined in TokenizerSetup of configuration. |
IsArray | bool | get/set | Is true for a possibly flattened array. |
Name | string | get/set | The name of the field. The name must be unique. |
Optional | bool | get/set | Optional field |
SampleValue | string | get/set | A short, display-ready sample of this field's content, captured from the first document
that carried a non-empty value for it. Surfaced asso a
caller configuring the field can see what it actually holds. |
PreloadFilters | bool | get/set | Setting this will make engine setup a valuefilter for each distinct value of a field.
The filters will be created and loaded during Load and may be accessed via
SearchEngine.GetPreloadedFilters. |
Type | JsonValueKind | get/set | Only primitive types and their arrays are accepted. Note that for fields containing
flattened arrays they will all have primitive types, and not JsonArray.
Use the IsArray property to check. |
Value | object | get/set | Actual value of the field. |
Weight | float | get/set | BM25PerFieldHow much this field contributes to the relevance score, relative to the other
searchable fields —in the BM25F formula, and the weight the per-field
scores are averaged by in.
Defaults to 1.0. Typical range: 1.0–3.0, where 1.0 is neutral and higher values
increase the field's contribution. Values below 1.0 de-emphasize the field.
Applies only to searchable fields; set it after Init() and before Index(). |
NameOfDocumentKeyField | string | get/set | Name of the key field used to identify documents.
Required for operations such as document deletion. |
AnalyzedDocumentCount | int | get | Number of documents the analyze pass (Init) saw in the stream. A value of 1 on a
large file is the classic "export envelope" symptom: a root object wrapping the
real document array, which the parser reads as one giant document. Callers can
use this to warn before Load/Index is attempted. |
Methods
Clone
Creates a deep copy of this field. The clone is detached: no NotifyChanged subscribers are carried over, and it can be changed without affecting the original.
Field Clone()Returns: Field
ValidateFieldValue
Validates that a value is compatible with the field's type and array requirements.
static bool ValidateFieldValue(Field field, object value, String& error)Fieldfield— The field definition to validate againstobjectvalue— The value to validateString&error— Output error message if validation fails
Returns: True if value is valid for the field, false otherwise
Dispose
void Dispose()GetFacetableFieldList
Returns the list of fields marked as Facetable.
List<Field> GetFacetableFieldList()Returns: List<Field>
GetField
Returns the field with the given name, or null if not found.
Field GetField(string fieldName)stringfieldName
Returns: Field
RequiresReindex
Returns true if the proposed configuration needsbefore it takes effect. Two quite different reasons, and both answer here so a caller has one question to ask:Filterable, Facetable, Sortable, Weight and PreloadFilters on a field that already has a role take effect at query time, and those still answer false.
bool RequiresReindex(FieldProxy[] proposed)FieldProxy[]proposed
Returns: bool
RequiresReload
Returns true when the proposal changes something a newhas to consume, socannot make it take effect however often it is called. Six flags are read by Load and nowhere else:,and(which decide the scoring components and the scoring mode),,(which fills the word matcher) and.
bool RequiresReload(FieldProxy[] proposed)FieldProxy[]proposed
Returns: bool
GetFieldList
Returns a list of all fields.
List<Field> GetFieldList()Returns: List<Field>
GetFilterableFieldList
Returns all fields marked as filterable.
List<Field> GetFilterableFieldList()Returns: List<Field>
GetSerialized
Returns the instance serialized as a JSON string.
string GetSerialized()Returns: string
Haskey
Returns true when this instance has a key field of the expected type.
bool Haskey()Returns: bool
SaveToFile
Serializes the field configuration to a file.
void SaveToFile(string fileName)stringfileName
Analyze
Analyzes a JSON string and extracts its field structure, repairing control characters and trailing commas on the way.
static DocumentFields Analyze(string jsonDocument, String& errorMessage)stringjsonDocumentString&errorMessage
Returns: DocumentFields
AnalyzeAsync
String@)Async variant of.
static Task<ValueTuple<DocumentFields, string>> AnalyzeAsync(Stream jsonStream)StreamjsonStream
Returns: Task<ValueTuple<DocumentFields, string>>
ReadFromFile
Deserializes an instance from a previously saved file.
static DocumentFields ReadFromFile(string fileName)stringfileName
Returns: DocumentFields
ScoringModeenum
Selects which BM25 scoring algorithm the engine uses when more than one field is marked as searchable. The mode is determined automatically attime from the per-fieldvalues:
| Name | Value |
|---|---|
BM25PerField | 0 |
BM25F | 1 |