Changelog

November - 2024

New features

• API Keys - now multiple api keys can be created and can be expired at any time. Individual key permissioning is coming soon
• Context keeping enhancements. We now natively support Hebrew, and added the ability to use an LLM for context keeping, opening the possibility to easily support any language. See platform Prefs / Context Detection
• Added support for Elastic's new "semantic" search type - in addition to lucene, hybrid, and vector
• mAIstro Updates:
  • Video! mAIstro now supports ingesting video via a new "Video Loop" node. You can easily pipeline a video to a multimodal LLM. See our example template "Video Loop"
  • PDF Loop - loop thru a pdf by page and automatically extract the text and take a snapshot image of the page for use in a multimodal LLM. Works great to get complicated information out of a business pdf. See the PDF Loop example template
  • OCR is now a callable node so you can use it as part of a backend process

October - 2024

New features

• mAIstro updates:
  • NTL now has code highlighting and rollup and a new editor
  • Code toolbox: a set of easy, single-node functions to:
  • extract generated code / sql / html from most LLMs,
  • protect, validate, and re-write SQL
  • Clean HTML and extract text
• HTML Cleanser updates. NeuralSeek automatically cleans scraped HTML docs in KB's that you connect. Now you can specify CSS selectors to remove on top of our normal cleansing, as well as disable the cleaner.
• Governance - Cost insights. Both sides of governance get a new tab that compares cost of your selected models across all other models we have capability and cost data for.
• DQL for elastic / watsonx discovery. We've brought our DQL interpreter over to elastic so you can pass DQL filters and easily do complicated filtering and lower migration risks when coming from discovery.

September - 2024

New features

• Multi-agent visual builder. Turn on multi-agent on the config tab and easily build category-driven multi-agent flows. No coding required! Every node of the multi-agent tree can have its own configuration (kb, llm's, everything) and guardrails. We've also unified the Seek and mAIstro sides of the house so you can call both from either api.
  • Each node can be of a traditional "seek" type, or a new mAIstro-lead node. mAIstro nodes send intents that hit them to a default action instead of creating a new intent. This lets you do disambiguation or focus the user onto capabilities you have enabled - like opening trouble tickets or other mAIstro-lead actions
  • Any intent can directly run a mAIstro flow. So for a question like "what's the weather like today" you could call out to a mAIstro flow instead of sending that to the traditional seek path/kb.
  • You can now directly add a new intent via the config and curate tabs.
  • Guardrails can run mAIstro flows. Min confidence, Min words and max words all can run custom mAIstro flows to give contextually & language -specific responses when those guardrails are hit.
• Chat! We've introduced a ChatGPT-like interface, as well as an SDK and embed code. You can quickly add a virtual agent to your website just by dropping in the embed code. The Chat SDK allows for drag & drop image and file operations - so you could easily build a bot that allows users to ask questions by providing images, such as "I want a refrigerator like this one"
• OCR! We soft-launched our OCR capabilities a few weeks back but never announced it. We now have OCR embedded into the system. When you use or document loader or upload a file into mAIstro - we'll automatically OCR any pdf's we find that are image-based and not text. You can also OCR image files. In addition we released a "dual ocr" template - basically showing you how to leverage and parralelize our OCR in addition to visual capabilities of a multi-modal model to do some amazing things for OCR'ing complex documents while keeping source formatting. This capability just blows away legacy OCR tools, with almost no tuning.
• Document Generation - we have built a new document generation behind the scenes, enabling greater capability in generating well-formatted PDF and Word files at scale.
• LLM's! LLama3.2 on both bedrock and watsonx.ai. This is the first multi-modal model available on watsonx.
• new curated model! For PPA plans curated model 1.1 is available in all regions
• Logging updates. Seek Logs have moved into the Governance tab, and include even more details. Enable corporate logging for transactional replay - which has quickly become a must-have for business SME's
• new mAIstro nodes! We released an XML toolkit, as well as about a dozen other new nodes & connectors
• translation enhancements! Via the api you can override the max Chunk size we use - which can dramatically speed up translations for mid-length translations when using slow LLM's / inference platforms.

August - 2024

New features

• mAIstro Min Confidence - In a seek, when hitting your min confidence threshold, run a custom mAIstro flow. You can use this to simply create a contextually aware "I don't know note" - but can also use this to kick off a notification, or escalation or externals service call or ticket... Anything really.
• Semantic Insights - now on the "hallucinated terms" chart you can click on a term to directly allow-list items...
• Data Loader - Drag & drop files to our new loader, which leverages mAIstro to chunk/load docs. You can use any mAIstro function or integration, make rest calls, generate embeddings, automatically loop and chunk documents... We give an example loader for elastic/watsonx discovery.
• Governace for mAIstro!
  • Automatically track and provide insights for all mAIstro templates, filterable by template.
  • Flow insights helps track time spent across our parallel engine, helping you optimize flows and understand where they are spending the most of their time.
  • Token insights mirrors the Seek token insights tab, helping show token consumption, cost, and model comparison options for the LLM's used to power your mAIstro flows
• Seek Governance updates
  • Filter by filter... When using filters in seek, now you can automatically track governance by the applied filter.

July - 2024

New features

• New LLM's
  • Mistral-large on watsonx.ai
  • GPT-4o-mini on OpenAI.
• Streaming api endpoints for seek and mAIstro for watsonx assistant. These have the required content type in the openAPI spec. Note- at the moment streaming seek is not recommended as you can’t use confidence scoring, nor get any payload fields like url. We’ll be working on this with the Watson team. New embedding models, and the ability to use a custom embedding model with NS intent detection and mAIstro
• Translation Improvements! NS translation is now up to 80% faster for large translations.
• NeuralSeek Hosted LLM's. When using a BYO-LLM plan we now provide a globally-hosted base LLM (mistral-7b) and purpose-built translation LLM for use with that plan for no additional charges, just the normal seek charge applies. THis should make it much easier to get started with NS.

June - 2024

New features

• New platforms supported:
  • vLLM / "generic" openAI-style inference engines. This allows you to plug-and play with many more on-prem and SaaS inference engines
  • Google Vertex is now supported, and we have added Gemini 1.5 pro and Flash. These models are quite good - Pro is on the same tier as GPT-4o, Claude3 Sonnet, and Mistral-Large
• mAIstro updates!
  • Built-in charting. With a compatible LLM you can ask for a chart to be generated as part of the output.
  • Formatted output - generate and display HTML and javascript
  • New "Raw" view - see the code behind charting and generated HTML
  • PDF output
  • Hover Menus! When on the visual builder, all of the nodes now will let you see and insert any secrets, user & system defined variables or generate a new variable. Makes building so much easier!
• Native integration to watsonx.governance. In mAistro, see our example template for how to configure this - it's really easy, just 3 steps. For watsonx.governance you just need an IAM API key, and from your "Production" deployment space in x.gov, under actions/model information we need your Evaluation datamart ID and Subscription ID. We'll send all of the NeuralSeek measures over to watsonx.governance so you can collect and govern them cross-instance and show a larger governance story. We also provide an open-ended integration in case you want to do something more custom.
• New mAIstro Integrations: (there are so many native functions and connectors in mAistro now we had to add a search feature!)
  • Jira
  • Trello
  • Github
  • Slack
  • AWS S3
  • Google/Bing/Yahoo/DuckDuckGo web searches.
• JSON Tools: We added JSON array filter and JSON Escape to make working with complicated payloads much easier inside mAIstro.
• Auto-Escaping. Now when using the mAIstro visual editor we will auto-escape any quotes. This should make building in mAIstro much easier for business users. We've found these updates plus the mAIstro auto-builder we released last month bring many usecases down to working "out of the box" with no additional modifications required to the autogenerated flows.
• Governance updates: We've enhanced the Token Insights tab, and added a new chart "Question Resolution" to the Overview tab to help track how many questions are hitting your minimum confidence threshold.
• The Logs tab now flags responses that had PII, HAP activation, and Prompt injection actions.

May - 2024

New features

• Virtual KB's! You can now use mAIstro to define a flow and use it as a virtual knowledgebase. Want to query multiple discovery instances at once? Easy. Elastic and DB2 and merge the results? Easy. Scrape a few webpages live and use those? Easy. See the new template in mAIstro for an example of how to configure this.
• Semantic Allow-list (Config / Semantic Model Tuning). Specify words or phrases to exclude from semantic penalties.
• Curate updates. Now answers generated by use of a filter will display the filter used during generation
• Custom Translations. Upload a training file via the API.

mAIstro Features

• Image processing / multimodal support in mAIstro. You can now grab images from the web, local file, or Google Docs and flow them thru LLM's that support image processing (Claude3, GPT-4, GPT-4o). See the new example template. And yes, you can power Seek based on images if you use this with the virtual KB!
• Auto-builder for mAIstro (SaaS - only). Have you been overwhelmed or afraid to try mAIstro? Not clear on how to build something? Now the welcome modal (and Load modal) will ask you to just describe your usecase, and then we'll auto-generate you a custom template.
• Snowflake connector! Now available in mAIstro

Governance Features

• Token Insights! A new module comes to NeuralSeek Governance (BYO-LLM plans only). Get cost insights on your LLM usage, metrics on generation speed, Cost comparisons to LLM's of similar capability. It's very compelling.
• Governance updates - now you can track cache and edited answer hit percentage from the Semantic Insights tab.

New Models

• Lots of new ones. GPT-4o, Mixtral8x-22, and more.

April - 2024

The launch of NeuralSeek Governance.

New features

• Remove Hallucinations - turn this on via the Configure tab under Semantic Scoring.  As part of a Seek response, remove any sentence containing a key word (proper noun, entity) that is not contained in your source documentation.
• Proposals. Our take on versioning / configuration changes.  You can now define a configuration as a "Proposal" and then call that proposal dynamically from the api or the Seek tab or Home tabs.  This helps separate admin configuration from SME's testing proposed changes.  It also lets you run multiple configs at once without passing a full override every time.  Update a config, and click "Propose Changes" In addition, a new feature "Log Alternate Configs" - lets you block the curation of answers coming from these propsals, so you can test in isolation in a single instance. Configuration Title and Description - as part of our Governance module and the launch of proposals we'll now as you for a configuration title and description on saving.  These flow into the governance side of the house for explainability.
• Pinecone support - our initial release.  more embedding model options are coming shortly.
• Milvus KB conector. So you can now do vector search into watsonx.data
• Return full Docs - we are rolling out the ability for you to return a full document instead of a passage.  Currently release for Discovery and AppSearch.  This way if you have carefully created or pre-snipped your documentation you can ensure the full document comes back.
• Performance improvements - some big updates on areas such as dynamic webscraping, context window splitting, and more.

mAIstro Features

• Secrets! - define variables on the Configure tab to hide them from normal mAIstro users.  On prem users can also define variables at the OS level. Very useful for passing / hiding DB connection info.
• Context Loop - split a large block of text by tokens and loop over it. Ver useful for translating large documents, or sending big things thru a small LLM.  See the Document Translation example in mAIstro
• Google Drive connector - pull from and write to a google drive
• Variable Loop - loop over an array of data

Governance Features

• Governance module.  Our initial focus with this first release is a holistic view of RAG governance with time-based and Intent/Category filtering. We'll be rolling out many more additional capability in the weeks to come here. At launch we have:
  • Executive overview charts
  • Intent Analytics - what intents are trending, and how are they performing - model / document regression
  • System Performance - monitor your instance and compare to the NS universe
  • Semantic insights - What is the quality of the answers being generated
  • Documentation Insights - What documentation is most used, and how is it performing
  • Configuration Insights - monitor configuration changes and track churn over time

New Models

• LLama 3 - a big step up from llama 2 in terms of its ability to follow directions.  In watsonx the context window is small, however so mixtral is still overall better.
• jais-13b-chat - in watsonx frankfurt, for Arabic usecases
• granite-7b-lab - This one seems better than the other granite models. Under the covers it's based on llama-2...
• Mistral-Large - similar and iteratively better than mixtral. not yet available on watsonx.

March - 2024

Explore is now renamed mAIstro and has gained a variety of new features.

New features

• Fully-custom RAG now available in NeuralSeek, offering simplicity via Seek and complexity via mAIstro, all out of the box and no-code required.

mAIstro Features

• Curate: Send your own Q&A into the curate, analytics, and log tabs.
• Categorize: Hook into the NS categorizer to get category and intent.
• Query Cache: Check for and return curated and edited answers.
• Semantic Score: Access the semantic scoring model from within a Maistro flow.
• Extract Grammar: Extract entities, nouns, dates, and more from text.
• Add Context: Recall the last turn of the conversation and inject the previous subject into text (for a KB or LLM call).
• Stop: Stop execution (useful for conditionals).
• Truncate by Tokens: Trim text by a set number of LLM tokens (use this to chop your KB documentation down to fit the LLM context window).

New Models

• Two new models added to watsonx in NeuralSeek: Granite 7B Japanese and Elyza Japanese Llama.

Other Updates

• New intro walk-me added to help new users get started on mAIstro.

 

February - 2024

New features

• Pre-LLM PII filtering/masking: Remove or mask personally identifiable information (PII) before sending queries to a Knowledge Base (KB) or LLM. Use pre-built elements or add your own using regular expressions.
• Prompt Injection detection: User input is scored against an internal model to identify potential prompt injection attempts. Problematic words are filtered out, and the entire input can be blocked based on the probability of prompt injection.
• Cross-language KB translation: When specifying a desired output language different from the KB language, user input can now be automatically translated into the KB language for better answers.
• Arbitrary Schemas for Explore: NeuralSeek Explore now supports arbitrary schemas, allowing users to hook it up to anything that sends a POST request, process it, and return it in the correct format. This feature enables dynamic rewording of messages based on saved context, chat history, or other criteria, providing a more personalized experience for users.
• Updates to Prompt Injection Mitigation: The try-it-out feature now displays scores of different phrases eligible to be removed from user input, enhancing the prompt injection detection capabilities.

New Models

• watsonx.ai introduces Granite-20b-5lang-instruct-rc model in tech preview, and several new models are added to Bedrock.

Explore Enhancements

• Guardrails such as Profanity Filter and Prompt Injection are now available in Explore.
• Several new example templates have been added to demonstrate these new features.
• Users can now modify the "WA Personalization" template provided in the examples on the Explore tab to dynamically reword messages flowing through Explore from Watson Assistant, offering a more personalized chatbot experience.
• The header parameters overrideschema and templatename in the explore API allow for easy configuration and customization of schemas in Explore, enabling seamless integration with various systems and applications.

 

January - 2024

New features

• Parallel "threaded" execution jobs introduced in Explore allow for faster execution of complicated templates, often outperforming custom-coding in Python.
• Enhancements to multi-turn seek: Users can now control the number of previous turns sent to the LLM for a more ChatGPT-style experience.
• Extract Enhancements:
  • Support for defining regex and keyword entity types, reducing workload on smaller/less capable LLMs and improving extraction speed.

Explore Enhancements

• Direct connectors to various databases including Postgres, Oracle, MySQL, MariaDB, MS SQL, and Redshift.
• System variables for injecting date, time, UUIDs, random numbers, etc.
• 'Extract' functionality added to Explore.
• Improved Explore OpenAPI template generator for easier integration with Watsonx Assistant.
• New templates available, including Custom RAG, Insurance Cause of Loss, and Conditional Logic.
• Option to specify the LLM to use in Explore LLM steps to avoid hitting rate limits and distribute the load effectively.

Updates

• Finer-grain user permissions: Users can now grant tab access while restricting write ability from specific tabs.
• All languages are now unlocked, allowing users to utilize NeuralSeek with any language supported by their chosen LLM.
• Stop/Cancel functionality for Seek and Explore during streaming responses.

 

December - 2023

New features

• Multilingual chain-of-thought prompting to enhance smaller LLMs like Llama and Granite for non-English languages.
• ElasticSearch / Watsonx Discovery Vector Search setup for hybrid or full vector search capabilities.
• KB ReRanker for custom result prioritization by field/tag and value lists.
• Profanity Filter implemented for multi-language profanity and hate speech filtering across all LLMs.
• Role-based access control for managing user permissions within the NeuralSeek UI.
• Explore enhancements:
  • OpenAPI spec generator for easy integration with Watson Assistant.
  • Inspector tool for debugging the Explore flow and variable states.
  • REST connector for making various HTTP requests and auto-parsing JSON into variables.
  • JSON to Variables stage for automatic variable creation from JSON input.
  • Output Variables formatting to match input parameters for seamless chaining in Explore.
  • Import/Export functionality for sharing templates across instances.
  • New functionality:
  • DB2 database connector
  • Table Prep (convert tables into natural language statements)
  • KB search filters
  • Stump for Seek (to sideload trusted data)
  • Regex
  • Several new example templates

New integrations

• Added Llama-2-chat Portuguese 13B to Watsonx Tech Preview.
• Release of Granite V2 in the model cards, offering improved performance over V1.

Updates

• Watsonx.ai models transitioned to streaming for improved timeout handling.
• Enhanced error reporting in the UI for Knowledge Bases (KBs) to show more detailed configuration feedback.
• Semantic Scoring model improvements with lemmatization consideration for partial match scoring.
• Watsonx Discovery automatic API key generation for simplified access.

 

November - 2023

New features

• Explore:
  • Expanded NTL-based explore functionality with drag-and-drop simplicity for building Explore routines.
  • Added the ability to create and save templates within the UI.
  • Introduced variables for easy API calling by passing template name and variable values.
  • Dynamic Variable Setting - Introduce the ability to dynamically set variables within a chain or flow, capture outputs into variables for endless reuse, and return all variables via the API (multi-output capability).
  • Recursion / Chained Explore - Enabled the creation of small, repeatable task templates that can be called from other explore templates, with shared variable memory space across templates, facilitating the creation of complex flows with ease.
  • New functionality:
  • Math Equations - Implemented full graphing-calculator level equations, overcoming the LLM's limitations with math by allowing users to set variables with LLMs, perform calculations in the math node, and then provide correct answers back into the LLM.
  • Force Numeric - Added a feature to extract numbers from text, ensuring that when a number is requested from the LLM, a numeric response is provided.
  • Split - Automated the removal of document headers and footers, enabling users to extract the content they need with ease.
  • POST - Provided the ability to call any REST service to submit data or initiate a downstream process.
  • Email - Introduced the functionality to send the output of a flow or variable content directly via email.

Updates

• Semantic Details on Seek - Unveiled the math behind the semantic score through a new modal on the seek tab, previously exclusive to API/developer use.
• Enhanced context keeping and semantic score for improved abilities in Spanish.
• Rolled out a new Spanish micro-model to assist with Spanish NLP.
• Updated base weights and prompting to counter GPT's recent drifting.
• Semantic Scoring now has the ability to consider document title and URL, capturing unique words that may be missing in the document itself.
• Added the ability to pass a filter column for regression testing.

 

October - 2023

New features

• "Generate Data" options in Explore tab – Send to LLM, Table Understanding
• "Logs" tab - See history of questions/answers given
• Hyper-personalization (Corporate document filtering)
• Corporate Logging - Connect NeuralSeek to an ElasticSearch instance to log everything around Seek, updates, edits, changes
• Configuration Logs - History of changed settings
• Enhancements to Explore:
  • "Seek" data
  • PII removal
  • Table Understanding

New integrations

• Elastic Search integration
• Multi-Turn Conversation Generation for Cognigy
• Mistral 7B Model support

Updates

• Released On-Prem "Flex" plan
• Added version numbering to "Integrate" tab sidebar
• Seek tab - "Show generated" option when the minimum confidence is not met

 

September - 2023

New features

• Explore: An Open-Ended Retrieval Augmented Generation Playground
• Vector Similarity for Intent Matching

New integrations

• Kore.ai Round Trip Monitoring
• IBM watsonx Granite Models Supported
• AWS Bedrock Integration / Models Supported
• Llama 2 Chat Model Support
• OpenSearch Integration
• HuggingFace Integration for Supported Models

Updates

• Refinements to Vector Similarity Matching

 

August - 2023

New features

• BYO-LLM plans – IBM watsonx language translation
• Option for summarization of document passage results from KB
• Option for Link Summarization of NeuralSeek Results, 1-5 Result Links
• 'Bring Your Own' Large Language Model (BYO-LLM) cards – ability to use multiple LLMs for a specific task

New integrations

• IBM Watson Assistant Dialog Multi-Turn Conversation Templates
• AWS Kendra Integration
• AWS Lex Multi-Turn Conversation Generation Templates

Updates

• New ‘Seek’ Parameter Call to Indicate LLM Preference
• Ability to set specific language on each LLM – e.g., “use THIS model for Spanish Seek / Translation”

 

July - 2023

New features

• Slot Filler - Ability to auto-fill slots when gathering information
• Offline spreadsheet editing with upload to Curate tab
• ConsoleAPI under Integrate tab
• Answer Streaming – users can now enable streaming responses from NeuralSeek with supported LLMs
• Translate Endpoint
• Curate to CSV / Upload Curated QA from CSV
• On-Prem deployment support
• New 'Identify Language' Endpoint
• Entity Extraction feature - Custom Entity Creation

New integrations

• IBM watsonx Model Compatibility
• AWS Lex Round-Trip Monitoring

Updates

• KnowledgeBase translation updated – questions now get translated to KnowledgeBase source language for summarization
• Cross-lingual support when using language code “xx” (Match Input) enhanced
• Semantic Match Analysis to describe the logic for the Semantic Score enhanced

 

June - 2023

New integrations

• IBM watsonx (LLM) connector

Updates

• AWS Partnership Announcement
• Improvements to Caching
• Confidence and Coverage Score Graphs added to Curate tab

 

May - 2023

New features

• Analytics API endpoint
• Table Extraction model to enable answers from tabular data

Updates

• Data Cleanser for non-HTML enabled

 

April - 2023

New features

• New plan - 'Bring Your Own' Large Language Model (BYO-LLM)
• Semantic Score Model, Improved Provenance and Semantic Source Re-Rank

New integrations

• Curate answers to Kore.ai, Cognigy, AWS Lex

Updates

• IBM Frankfurt (FRA) data center availability
• IBM Sydney (SYD) data center availability

 

March - 2023

New features

• Personal Identifiable Information (PII) Detection
• Sentiment Analysis
• Source Document Monitoring and Answer Regeneration

New integrations

• Watson Assistant Round-Trip Logging

Updates

• User-specified input length enabled

 

February - 2023

New features

• Personalization of generated answers

New integrations

• Auto-Build Watson Assistant Multi-Step Action

Updates

• Additional languages enabled (Chinese, Czech, Dutch, Indonesian, Japanese)
• Enhanced API to allow run-time modification of all parameters
• KB tuning parameters enabled
• Large Language Model (LLM) tuning