Skip to content

Transform

Summarize

Summarizes input text while preserving the main subject of the content.

{{ summarize|length:100|match:"" }}

Parameters

  • Length: The total maximum character length of the output/summary.
  • Match: The text around which to prioritize the summary.

Returns

The resulting summary.

Example Usage 1 
I have 20 cats and 40 dogs - it's a lot of furry friends to take care of! 
My name is Jane and I run an animal rescue shelter out of my home. 
It all started a few years ago when I took in a litter of abandoned kittens. 
I fell in love with them and decided to make it my mission to give unwanted animals a forever home. 
{{ summarize|length:100 }}

Yields:

I have 20 cats and 40 dogs - it's a lot of furry friends to take care of!
Example Usage 2 - Using match 
I have 20 cats and 40 dogs - it's a lot of furry friends to take care of! 
My name is Jane and I run an animal rescue shelter out of my home. 
It all started a few years ago when I took in a litter of abandoned kittens. 
I fell in love with them and decided to make it my mission to give unwanted animals a forever home. 
{{ summarize|length:100|match:"love" }}

Yields:

I fell in love with them and decided to make it my mission to give unwanted animals a forever home.

Translate

Translates input text into a language of your choice.

{{ translate  | target: "" }}

Parameters

  • Target: The language you wish to translate the text into. You must enter the language's 2-character ISO 639 code to get results. A full list of language codes can be found here.

Returns

The original text translated into the language of your choice.

Example Usage 
The translate function can easily translate text into any language you desire!
{{ translate  | target: "es" }}

Would yield:

La función de traducción puede traducir fácilmente el texto a cualquier idioma que desee.

Truncate by Tokens

Helpful to manage the size of context sent to the LLM, this allows you to truncate to a specific number of tokens effortlessly.

{{ truncateToken  | tokens: "" }}

Parameters

  • Tokens: The maximum number of tokens to allow in the returned text.

Returns

The resulting text clipped to the specified amount of tokens.

Example Usage 
{{ kb | query: "NeuralSeek" }}=>{{ truncateToken | tokens: "2000" }}=>{{ variable | name: "documentation" }}

Would yield a variable far too large to include here, but would limit the resulting documentation text to 2000 (2k) tokens before assigning to the documentation variable. This helps prevent exceeding context windows of some smaller LLMs.

Remove Stopwords

Removes stop words from input text.

{{ stopwords }}

Parameters

None - Data should be "chained" into this function.

Returns

The resulting text with stopwords removed.

Example Usage 
I have 20 cats and 40 dogs, isn't this amazing?
{{ stopwords }}

Will yield

20 cats 40 dogs, amazing?

Notice the words I, have, and, isn't, this are deemed as stopwords and thus have been removed.

Force Numeric

This function removes all non-numeric characters, and string-style concatenates the remainder into a single value.

{{ forceNumeric }}

Parameters

None - Data should be "chained" into this function.

Returns

The resulting number.

Example Usage

I have 20 cats and 40 dogs contains numeric values.  So, running this:

I have 20 cats and 40 dogs
{{ forceNumeric }}

Will yield: 2040

Table Prep

This function prepares tabular data to be better understood and processed by LLM.

{{ tablePrep | query:"" | sentences: "true" }}

Parameters

  • Query: Keywords to help narrow the returned data.
  • Sentences: If true, return the output in natural language expressions. If false, return JSON format.

Returns

The resulting natural language text or JSON.

Example Usage 1

If we have CSV data, table prep will convert it to JSON or natural language:

col1,col2,col3
data1,data2,data3
data11,data22,data33
{{ tablePrep | sentences: "false" }}

The result will be:

{
"col1": [
"data1",
"data11"
],
"col2": [
"data2",
"data22"
],
"col3": [
"data3",
"data33"
]
}
Example Usage 2 - Using the query parameter 
col1,col2,col3
data1,data2,data3
data11,data22,data33
{{ tablePrep|query: "values for col1" }}

Will yield all the values for col1:

{
"col1": [
"data1",
"data11"
]
}
Example Usage 3 - Using the sentences: true parameter 
col1,col2,col3
data1,data2,data3
data11,data22,data33
{{ tablePrep | sentences: "true" }}

Will yield:

Record number 0 lists that col1 is data1, and the col2 is data2, and the col3 is data3.
Record number 1 lists that col1 is data11, and the col2 is data22, and the col3 is data33.