Fun

News Feed - 2023-11-21 02:11:44

Tristan Greene6 hours agoScientists develop AI monitoring agent to detect and stop harmful outputsThe monitoring system is designed to detect and thwart both prompt injection attacks and edge-case threats.2877 Total views10 Total sharesListen to article 0:00NewsJoin us on social networksA team of researchers from artificial intelligence (AI) firm AutoGPT, Northeastern University and Microsoft Research have developed a tool that monitors large language models (LLMs) for potentially harmful outputs and prevents them from executing. 


The agent is described in a preprint research paper titled “Testing Language Model Agents Safely in the Wild.” According to the research, the agent is flexible enough to monitor existing LLMs and can stop harmful outputs, such as code attacks, before they happen.


Per the research:“Agent actions are audited by a context-sensitive monitor that enforces a stringent safety boundary to stop an unsafe test, with suspect behavior ranked and logged to be examined by humans.”


The team writes that existing tools for monitoring LLM outputs for harmful interactions seemingly work well in laboratory settings, but when applied to testing models already in production on the open internet, they “often fall short of capturing the dynamic intricacies of the real world.”


This, seemingly, is because of the existence of edge cases. Despite the best efforts of the most talented computer scientists, the idea that researchers can imagine every possible harm vector before it happens is largely considered an impossibility in the field of AI.


Even when the humans interacting with AI have the best intentions, unexpected harm can arise from seemingly innocuous prompts.An illustration of the monitor in action. On the left, a workflow ending in a high safety rating. On the right, a workflow ending in a low safety rating. Source: Naihin, et., al. 2023


To train the monitoring agent, the researchers built a data set of nearly 2,000 safe human-AI interactions across 29 different tasks ranging from simple text-retrieval tasks and coding corrections all the way to developing entire webpages from scratch.


Related:Meta dissolves responsible AI division amid restructuring


They also created a competing testing data set filled with manually created adversarial outputs, including dozens intentionally designed to be unsafe.


The data sets were then used to train an agent on OpenAI’s GPT 3.5 turbo, a state-of-the-art system, capable of distinguishing between innocuous and potentially harmful outputs with an accuracy factor of nearly 90%.# Microsoft# AI# ChatGPTAdd reactionAdd reactionRead moreHow blockchain, AI can help research into extending human lifeScammers play a long game using bogus, AI-backed "law firm"Google to invest another $2B in AI firm Anthropic: Report

News Feed

Tokenization to unlock interoperability across payments, investments
Ana Paula Pereira6 hours agoTokenization to unlock interoperability across payments, investmentsDuring the TokenizeThis 2024 event, executives from Ripple and Stellar discussed the latest trends in tokenization, includin
William Suberg7 hours ago100%+ BTC price gains? Bitcoin faces ‘massively overvalued’ stocksBitcoin posted a classic “Uptober,” but risk assets across the board risk a serious contraction, forecasts warn.3817 Tota
Chainlink Whales Dump Over 170 Million LINK In Three Weeks – Selling Pressure Ahead?
Reason to trust Strict editorial policy that focuses on accuracy, relevance, and impartiality Created by industry experts and meticulously reviewed The highest standards in reporting and pu
US Lawmaker Suggests ‘Maybe’ Crypto Should Be Banned Citing Bigger Issues Than FTX
US Lawmaker Suggests "Maybe" Crypto Should Be Banned Citing Bigger Issues Than FTX A U.S. senator has suggested that cryptocurrency should “maybe” be banned following t
NFT Economy Grows Exponential: $1M in Non-Fungible Token Sales Last Week
NFT Economy Grows Exponential: $1M in Non-Fungible Token Sales Last WeekWhile a number of people are focused on decentralized finance (defi), the non-fungible token (NFT) industry h
Biggest Bank in El Salvador Now Accepts Bitcoin as Payment for Financial Products
Biggest Bank in El Salvador Now Accepts Bitcoin as Payment for Financial Products Bancoagricola, the biggest bank in El Salvador, is now accepting bitcoin to pay for debts originat
After Blockstack’s Regulated Offering, Where Now For US Token Sales?
After Blockstack’s Regulated Offering, Where Now For US Token Sales? When Blockstack announced the first Reg A+ token sale, many believed it would open the floodgates for a spa
81.79 ‘Sleeping Bitcoin’ From 2011 Worth $3.6M Moved for the First Time in Over a Decade
81.79 "Sleeping Bitcoin" From 2011 Worth $3.6M Moved for the First Time in Over a Decade As bitcoin has increased more than 5% in value against the U.S. dollar during the last week
While BTC Skyrocketed to $69K, Whale From 2013 Transfers $147 Million Worth of ‘Sleeping Bitcoins’
While BTC Skyrocketed to $69K, Whale From 2013 Transfers $147 Million Worth of "Sleeping Bitcoins" Following the string of 20 block rewards spent on Wednesday, an idle bitcoin wall
Full Ban on Crypto in Russia Would Be Counterproductive, Rosfinmonitoring Says
Full Ban on Crypto in Russia Would Be Counterproductive, Rosfinmonitoring Says Russian citizens and businesses already own cryptocurrencies, which is why a complete crypto ban woul
Crypto Exchange Binance Approved by French Regulator as a Fully Regulated Digital Asset Service Provider
Crypto Exchange Binance Approved by French Regulator as a Fully Regulated Digital Asset Service Provider Cryptocurrency exchange Binance has received regulatory approval from the F
If SEC approves spot Ether ETFs, many ‘will be caught severely offside’
Ciaran Lyons1 hour agoIf SEC approves spot Ether ETFs, many ‘will be caught severely offside’Coinbase institutional research analyst David Han believes “there is room for surprise to the upside on this decision.”