June - 2024

New features

• New platforms supported:
  • vLLM / "generic" openAI-style inference engines. This allows you to plug-and play with many more on-prem and SaaS inference engines
  • Google Vertex is now supported, and we have added Gemini 1.5 pro and Flash. These models are quite good - Pro is on the same tier as GPT-4o, Claude3 Sonnet, and Mistral-Large
• mAIstro updates!
  • Built-in charting. With a compatible LLM you can ask for a chart to be generated as part of the output.
  • Formatted output - generate and display HTML and javascript
  • New "Raw" view - see the code behind charting and generated HTML
  • PDF output
  • Hover Menus! When on the visual builder, all of the nodes now will let you see and insert any secrets, user & system defined variables or generate a new variable. Makes building so much easier!
• Native integration to watsonx.governance. In mAistro, see our example template for how to configure this - it's really easy, just 3 steps. For watsonx.governance you just need an IAM API key, and from your "Production" deployment space in x.gov, under actions/model information we need your Evaluation datamart ID and Subscription ID. We'll send all of the NeuralSeek measures over to watsonx.governance so you can collect and govern them cross-instance and show a larger governance story. We also provide an open-ended integration in case you want to do something more custom.
• New mAIstro Integrations: (there are so many native functions and connectors in mAistro now we had to add a search feature!)
  • Jira
  • Trello
  • Github
  • Slack
  • AWS S3
  • Google/Bing/Yahoo/DuckDuckGo web searches.
• JSON Tools: We added JSON array filter and JSON Escape to make working with complicated payloads much easier inside mAIstro.
• Auto-Escaping. Now when using the mAIstro visual editor we will auto-escape any quotes. This should make building in mAIstro much easier for business users. We've found these updates plus the mAIstro auto-builder we released last month bring many usecases down to working "out of the box" with no additional modifications required to the autogenerated flows.
• Governance updates: We've enhanced the Token Insights tab, and added a new chart "Question Resolution" to the Overview tab to help track how many questions are hitting your minimum confidence threshold.
• The Logs tab now flags responses that had PII, HAP activation, and Prompt injection actions.

May - 2024

New features

• Virtual KB's! You can now use mAIstro to define a flow and use it as a virtual knowledgebase. Want to query multiple discovery instances at once? Easy. Elastic and DB2 and merge the results? Easy. Scrape a few webpages live and use those? Easy. See the new template in mAIstro for an example of how to configure this.
• Semantic Allow-list (Config / Semantic Model Tuning). Specify words or phrases to exclude from semantic penalties.
• Curate updates. Now answers generated by use of a filter will display the filter used during generation
• Custom Translations. Upload a training file via the API.

mAIstro Features

• Image processing / multimodal support in mAIstro. You can now grab images from the web, local file, or Google Docs and flow them thru LLM's that support image processing (Claude3, GPT-4, GPT-4o). See the new example template. And yes, you can power Seek based on images if you use this with the virtual KB!
• Auto-builder for mAIstro (SaaS - only). Have you been overwhelmed or afraid to try mAIstro? Not clear on how to build something? Now the welcome modal (and Load modal) will ask you to just describe your usecase, and then we'll auto-generate you a custom template.
• Snowflake connector! Now available in mAIstro

Governance Features

• Token Insights! A new module comes to NeuralSeek Governance (BYO-LLM plans only). Get cost insights on your LLM usage, metrics on generation speed, Cost comparisons to LLM's of similar capability. It's very compelling.
• Governance updates - now you can track cache and edited answer hit percentage from the Semantic Insights tab.

New Models

• Lots of new ones. GPT-4o, Mixtral8x-22, and more.

April - 2024

The launch of NeuralSeek Governance.

New features

• Remove Hallucinations - turn this on via the Configure tab under Semantic Scoring.  As part of a Seek response, remove any sentence containing a key word (proper noun, entity) that is not contained in your source documentation.
• Proposals. Our take on versioning / configuration changes.  You can now define a configuration as a "Proposal" and then call that proposal dynamically from the api or the Seek tab or Home tabs.  This helps separate admin configuration from SME's testing proposed changes.  It also lets you run multiple configs at once without passing a full override every time.  Update a config, and click "Propose Changes" In addition, a new feature "Log Alternate Configs" - lets you block the curation of answers coming from these propsals, so you can test in isolation in a single instance. Configuration Title and Description - as part of our Governance module and the launch of proposals we'll now as you for a configuration title and description on saving.  These flow into the governance side of the house for explainability.
• Pinecone support - our initial release.  more embedding model options are coming shortly.
• Milvus KB conector. So you can now do vector search into watsonx.data
• Return full Docs - we are rolling out the ability for you to return a full document instead of a passage.  Currently release for Discovery and AppSearch.  This way if you have carefully created or pre-snipped your documentation you can ensure the full document comes back.
• Performance improvements - some big updates on areas such as dynamic webscraping, context window splitting, and more.

mAIstro Features

• Secrets! - define variables on the Configure tab to hide them from normal mAIstro users.  On prem users can also define variables at the OS level. Very useful for passing / hiding DB connection info.
• Context Loop - split a large block of text by tokens and loop over it. Ver useful for translating large documents, or sending big things thru a small LLM.  See the Document Translation example in mAIstro
• Google Drive connector - pull from and write to a google drive
• Variable Loop - loop over an array of data

Governance Features

• Governance module.  Our initial focus with this first release is a holistic view of RAG governance with time-based and Intent/Category filtering. We'll be rolling out many more additional capability in the weeks to come here. At launch we have:
  • Executive overview charts
  • Intent Analytics - what intents are trending, and how are they performing - model / document regression
  • System Performance - monitor your instance and compare to the NS universe
  • Semantic insights - What is the quality of the answers being generated
  • Documentation Insights - What documentation is most used, and how is it performing
  • Configuration Insights - monitor configuration changes and track churn over time

New Models

• LLama 3 - a big step up from llama 2 in terms of its ability to follow directions.  In watsonx the context window is small, however so mixtral is still overall better.
• jais-13b-chat - in watsonx frankfurt, for Arabic usecases
• granite-7b-lab - This one seems better than the other granite models. Under the covers it's based on llama-2...
• Mistral-Large - similar and iteratively better than mixtral. not yet available on watsonx.

March - 2024

Explore is now renamed mAIstro and has gained a variety of new features.

New features

• Fully-custom RAG now available in NeuralSeek, offering simplicity via Seek and complexity via mAIstro, all out of the box and no-code required.

mAIstro Features

• Curate: Send your own Q&A into the curate, analytics, and log tabs.
• Categorize: Hook into the NS categorizer to get category and intent.
• Query Cache: Check for and return curated and edited answers.
• Semantic Score: Access the semantic scoring model from within a Maistro flow.
• Extract Grammar: Extract entities, nouns, dates, and more from text.
• Add Context: Recall the last turn of the conversation and inject the previous subject into text (for a KB or LLM call).
• Stop: Stop execution (useful for conditionals).
• Truncate by Tokens: Trim text by a set number of LLM tokens (use this to chop your KB documentation down to fit the LLM context window).

New Models

• Two new models added to watsonx in NeuralSeek: Granite 7B Japanese and Elyza Japanese Llama.

Other Updates

• New intro walk-me added to help new users get started on mAIstro.

 

February - 2024

New features

• Pre-LLM PII filtering/masking: Remove or mask personally identifiable information (PII) before sending queries to a Knowledge Base (KB) or LLM. Use pre-built elements or add your own using regular expressions.
• Prompt Injection detection: User input is scored against an internal model to identify potential prompt injection attempts. Problematic words are filtered out, and the entire input can be blocked based on the probability of prompt injection.
• Cross-language KB translation: When specifying a desired output language different from the KB language, user input can now be automatically translated into the KB language for better answers.
• Arbitrary Schemas for Explore: NeuralSeek Explore now supports arbitrary schemas, allowing users to hook it up to anything that sends a POST request, process it, and return it in the correct format. This feature enables dynamic rewording of messages based on saved context, chat history, or other criteria, providing a more personalized experience for users.
• Updates to Prompt Injection Mitigation: The try-it-out feature now displays scores of different phrases eligible to be removed from user input, enhancing the prompt injection detection capabilities.

New Models

• watsonx.ai introduces Granite-20b-5lang-instruct-rc model in tech preview, and several new models are added to Bedrock.

Explore Enhancements

• Guardrails such as Profanity Filter and Prompt Injection are now available in Explore.
• Several new example templates have been added to demonstrate these new features.
• Users can now modify the "WA Personalization" template provided in the examples on the Explore tab to dynamically reword messages flowing through Explore from Watson Assistant, offering a more personalized chatbot experience.
• The header parameters overrideschema and templatename in the explore API allow for easy configuration and customization of schemas in Explore, enabling seamless integration with various systems and applications.

 

January - 2024

New features

• Parallel "threaded" execution jobs introduced in Explore allow for faster execution of complicated templates, often outperforming custom-coding in Python.
• Enhancements to multi-turn seek: Users can now control the number of previous turns sent to the LLM for a more ChatGPT-style experience.
• Extract Enhancements:
  • Support for defining regex and keyword entity types, reducing workload on smaller/less capable LLMs and improving extraction speed.

Explore Enhancements

• Direct connectors to various databases including Postgres, Oracle, MySQL, MariaDB, MS SQL, and Redshift.
• System variables for injecting date, time, UUIDs, random numbers, etc.
• 'Extract' functionality added to Explore.
• Improved Explore OpenAPI template generator for easier integration with Watsonx Assistant.
• New templates available, including Custom RAG, Insurance Cause of Loss, and Conditional Logic.
• Option to specify the LLM to use in Explore LLM steps to avoid hitting rate limits and distribute the load effectively.

Updates

• Finer-grain user permissions: Users can now grant tab access while restricting write ability from specific tabs.
• All languages are now unlocked, allowing users to utilize NeuralSeek with any language supported by their chosen LLM.
• Stop/Cancel functionality for Seek and Explore during streaming responses.

 

December - 2023

New features

• Multilingual chain-of-thought prompting to enhance smaller LLMs like Llama and Granite for non-English languages.
• ElasticSearch / Watsonx Discovery Vector Search setup for hybrid or full vector search capabilities.
• KB ReRanker for custom result prioritization by field/tag and value lists.
• Profanity Filter implemented for multi-language profanity and hate speech filtering across all LLMs.
• Role-based access control for managing user permissions within the NeuralSeek UI.
• Explore enhancements:
  • OpenAPI spec generator for easy integration with Watson Assistant.
  • Inspector tool for debugging the Explore flow and variable states.
  • REST connector for making various HTTP requests and auto-parsing JSON into variables.
  • JSON to Variables stage for automatic variable creation from JSON input.
  • Output Variables formatting to match input parameters for seamless chaining in Explore.
  • Import/Export functionality for sharing templates across instances.
  • New functionality:
  • DB2 database connector
  • Table Prep (convert tables into natural language statements)
  • KB search filters
  • Stump for Seek (to sideload trusted data)
  • Regex
  • Several new example templates

New integrations

• Added Llama-2-chat Portuguese 13B to Watsonx Tech Preview.
• Release of Granite V2 in the model cards, offering improved performance over V1.

Updates

• Watsonx.ai models transitioned to streaming for improved timeout handling.
• Enhanced error reporting in the UI for Knowledge Bases (KBs) to show more detailed configuration feedback.
• Semantic Scoring model improvements with lemmatization consideration for partial match scoring.
• Watsonx Discovery automatic API key generation for simplified access.

 

November - 2023

New features

• Explore:
  • Expanded NTL-based explore functionality with drag-and-drop simplicity for building Explore routines.
  • Added the ability to create and save templates within the UI.
  • Introduced variables for easy API calling by passing template name and variable values.
  • Dynamic Variable Setting - Introduce the ability to dynamically set variables within a chain or flow, capture outputs into variables for endless reuse, and return all variables via the API (multi-output capability).
  • Recursion / Chained Explore - Enabled the creation of small, repeatable task templates that can be called from other explore templates, with shared variable memory space across templates, facilitating the creation of complex flows with ease.
  • New functionality:
  • Math Equations - Implemented full graphing-calculator level equations, overcoming the LLM's limitations with math by allowing users to set variables with LLMs, perform calculations in the math node, and then provide correct answers back into the LLM.
  • Force Numeric - Added a feature to extract numbers from text, ensuring that when a number is requested from the LLM, a numeric response is provided.
  • Split - Automated the removal of document headers and footers, enabling users to extract the content they need with ease.
  • POST - Provided the ability to call any REST service to submit data or initiate a downstream process.
  • Email - Introduced the functionality to send the output of a flow or variable content directly via email.

Updates

• Semantic Details on Seek - Unveiled the math behind the semantic score through a new modal on the seek tab, previously exclusive to API/developer use.
• Enhanced context keeping and semantic score for improved abilities in Spanish.
• Rolled out a new Spanish micro-model to assist with Spanish NLP.
• Updated base weights and prompting to counter GPT's recent drifting.
• Semantic Scoring now has the ability to consider document title and URL, capturing unique words that may be missing in the document itself.
• Added the ability to pass a filter column for regression testing.

 

October - 2023

New features

• "Generate Data" options in Explore tab – Send to LLM, Table Understanding
• "Logs" tab - See history of questions/answers given
• Hyper-personalization (Corporate document filtering)
• Corporate Logging - Connect NeuralSeek to an ElasticSearch instance to log everything around Seek, updates, edits, changes
• Configuration Logs - History of changed settings
• Enhancements to Explore:
  • "Seek" data
  • PII removal
  • Table Understanding

New integrations

• Elastic Search integration
• Multi-Turn Conversation Generation for Cognigy
• Mistral 7B Model support

Updates

• Released On-Prem "Flex" plan
• Added version numbering to "Integrate" tab sidebar
• Seek tab - "Show generated" option when the minimum confidence is not met

 

September - 2023

New features

• Explore: An Open-Ended Retrieval Augmented Generation Playground
• Vector Similarity for Intent Matching

New integrations

• Kore.ai Round Trip Monitoring
• IBM watsonx Granite Models Supported
• AWS Bedrock Integration / Models Supported
• Llama 2 Chat Model Support
• OpenSearch Integration
• HuggingFace Integration for Supported Models

Updates

• Refinements to Vector Similarity Matching

 

August - 2023

New features

• BYO-LLM plans – IBM watsonx language translation
• Option for summarization of document passage results from KB
• Option for Link Summarization of NeuralSeek Results, 1-5 Result Links
• 'Bring Your Own' Large Language Model (BYO-LLM) cards – ability to use multiple LLMs for a specific task

New integrations

• IBM Watson Assistant Dialog Multi-Turn Conversation Templates
• AWS Kendra Integration
• AWS Lex Multi-Turn Conversation Generation Templates

Updates

• New ‘Seek’ Parameter Call to Indicate LLM Preference
• Ability to set specific language on each LLM – e.g., “use THIS model for Spanish Seek / Translation”

 

July - 2023

New features

• Slot Filler - Ability to auto-fill slots when gathering information
• Offline spreadsheet editing with upload to Curate tab
• ConsoleAPI under Integrate tab
• Answer Streaming – users can now enable streaming responses from NeuralSeek with supported LLMs
• Translate Endpoint
• Curate to CSV / Upload Curated QA from CSV
• On-Prem deployment support
• New 'Identify Language' Endpoint
• Entity Extraction feature - Custom Entity Creation

New integrations

• IBM watsonx Model Compatibility
• AWS Lex Round-Trip Monitoring

Updates

• KnowledgeBase translation updated – questions now get translated to KnowledgeBase source language for summarization
• Cross-lingual support when using language code “xx” (Match Input) enhanced
• Semantic Match Analysis to describe the logic for the Semantic Score enhanced

 

June - 2023

New integrations

• IBM watsonx (LLM) connector

Updates

• AWS Partnership Announcement
• Improvements to Caching
• Confidence and Coverage Score Graphs added to Curate tab

 

May - 2023

New features

• Analytics API endpoint
• Table Extraction model to enable answers from tabular data

Updates

• Data Cleanser for non-HTML enabled

 

April - 2023

New features

• New plan - 'Bring Your Own' Large Language Model (BYO-LLM)
• Semantic Score Model, Improved Provenance and Semantic Source Re-Rank

New integrations

• Curate answers to Kore.ai, Cognigy, AWS Lex

Updates

• IBM Frankfurt (FRA) data center availability
• IBM Sydney (SYD) data center availability

 

March - 2023

New features

• Personal Identifiable Information (PII) Detection
• Sentiment Analysis
• Source Document Monitoring and Answer Regeneration

New integrations

• Watson Assistant Round-Trip Logging

Updates

• User-specified input length enabled

 

February - 2023

New features

• Personalization of generated answers

New integrations

• Auto-Build Watson Assistant Multi-Step Action

Updates

• Additional languages enabled (Chinese, Czech, Dutch, Indonesian, Japanese)
• Enhanced API to allow run-time modification of all parameters
• KB tuning parameters enabled
• Large Language Model (LLM) tuning