Connect with us

Hi, what are you looking for?


Nvidia publishes a “safer” text-generating AI toolbox.


Despite the hype, text-generating AI models like OpenAI’s GPT-4 make many blunders, some detrimental. James Vincent of The Verge labeled one model an “emotionally manipulative liar,” which sums up the present situation.

These models’ companies say they’re deploying filters and human moderators to remedy errors when they’re detected. But, unfortunately, there’s no proper answer. Biases, toxicity, and malicious assaults affect even the finest models.

Today, Nvidia unveiled NeMo Guardrails, an open-source toolkit to make AI-powered apps more “accurate, appropriate, on topic and secure” by improving text-generating models.

Nvidia’s VP of applied research, Jonathan Cohen, says the firm had been working on Guardrails’ core architecture for “many years” but just recognized it was a suitable fit for models like GPT-4 and ChatGPT a year ago.

Cohen informed TechCrunch via email that NeMo Guardrails has been in development since. Enterprise model deployment requires AI model safety tools.

Guardrails “safeguards” AI programs that create text and voice with code, examples, and documentation. Nvidia says the toolkit can construct rules in a few lines of code for most generative language models.

Guardrails can restrict models from going off-subject, replying with false or harmful information, and connecting to “unsafe” external sources. For example, keep a customer service assistant from answering whether queries or a search engine chatbot from linking to unreliable academic papers.

Cohen added Guardrails to let developers set their application’s boundaries. However, they may create too broad or narrow guardrails for their use case.

A universal cure for language model flaws sounds too wonderful to be true, and it is. Nvidia admits that Guardrails won’t capture everything, but firms like Zapier utilize it to make their generative models safer.

According to Cohen, guardrails work well with models “sufficiently good at instruction-following” like ChatGPT and employ the popular LangChain framework for AI-powered apps. But, unfortunately, that eliminates certain open-source choices.

Despite Guardrails’ efficacy, Nvidia isn’t offering it for charity. Nvidia’s corporate AI software package and fully managed cloud service use the NeMo framework, which includes it. As a result, Nvidia would rather companies pay for the hosted Guardrails version than the open-source version.

Guardrails are likely safe, but Nvidia should not imply otherwise.

Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *

The future of technological innovation is here. Be the first to discover the latest advancements, insights, and reviews. Join us in shaping the future.

You May Also Like


Every year, new gadgets are brought to the market as technology advances. The debut of some of the most cutting-edge and intriguing gadgets ever...


Are you ready to enter a world of limitless possibilities? Artificial Intelligence is here, and it's changing the game. From improved healthcare to increased...


Get ready to embrace the future with open arms - the top 10 tech trends of 2023 are about to revolutionize the world!"  ...


What distinguishes open-source software from proprietary software? See the pros and cons of this tech favorite. Pros: Open-source software is usually free or low-cost...