Fun

News Feed - 2023-11-21 02:11:44

Tristan Greene6 hours agoScientists develop AI monitoring agent to detect and stop harmful outputsThe monitoring system is designed to detect and thwart both prompt injection attacks and edge-case threats.2877 Total views10 Total sharesListen to article 0:00NewsJoin us on social networksA team of researchers from artificial intelligence (AI) firm AutoGPT, Northeastern University and Microsoft Research have developed a tool that monitors large language models (LLMs) for potentially harmful outputs and prevents them from executing. 


The agent is described in a preprint research paper titled “Testing Language Model Agents Safely in the Wild.” According to the research, the agent is flexible enough to monitor existing LLMs and can stop harmful outputs, such as code attacks, before they happen.


Per the research:“Agent actions are audited by a context-sensitive monitor that enforces a stringent safety boundary to stop an unsafe test, with suspect behavior ranked and logged to be examined by humans.”


The team writes that existing tools for monitoring LLM outputs for harmful interactions seemingly work well in laboratory settings, but when applied to testing models already in production on the open internet, they “often fall short of capturing the dynamic intricacies of the real world.”


This, seemingly, is because of the existence of edge cases. Despite the best efforts of the most talented computer scientists, the idea that researchers can imagine every possible harm vector before it happens is largely considered an impossibility in the field of AI.


Even when the humans interacting with AI have the best intentions, unexpected harm can arise from seemingly innocuous prompts.An illustration of the monitor in action. On the left, a workflow ending in a high safety rating. On the right, a workflow ending in a low safety rating. Source: Naihin, et., al. 2023


To train the monitoring agent, the researchers built a data set of nearly 2,000 safe human-AI interactions across 29 different tasks ranging from simple text-retrieval tasks and coding corrections all the way to developing entire webpages from scratch.


Related:Meta dissolves responsible AI division amid restructuring


They also created a competing testing data set filled with manually created adversarial outputs, including dozens intentionally designed to be unsafe.


The data sets were then used to train an agent on OpenAI’s GPT 3.5 turbo, a state-of-the-art system, capable of distinguishing between innocuous and potentially harmful outputs with an accuracy factor of nearly 90%.# Microsoft# AI# ChatGPTAdd reactionAdd reactionRead moreHow blockchain, AI can help research into extending human lifeScammers play a long game using bogus, AI-backed "law firm"Google to invest another $2B in AI firm Anthropic: Report

News Feed

Robinhood Q2 crypto revenue surges 161% on rising trading volume
Brayden Lindrea8 hours agoRobinhood Q2 crypto revenue surges 161% on rising trading volumeThe $81 million Robinhood made in crypto revenue was more than double made from equities in Q2.6999 Total views17 Total sharesList
Bitcoin Golden Cross In Play – Analyst Reveals Best Course Of Action
Reason to trust Strict editorial policy that focuses on accuracy, relevance, and impartiality Created by industry experts and meticulously reviewed The highest standards in reporting and pu
Trump weighing Bitcoin hater Jamie Dimon for US Treasury
Tristan Greene7 hours agoTrump weighing Bitcoin hater Jamie Dimon for US TreasuryThe former president could reveal more details at the Bitcoin Conference in Nashville from July 25 through 27.3465 Total views20 Total shar
Zhiyuan Sun8 hours agoChinese man sentenced to 9 months in prison for buying $13K in USDTChina has harshly enforced its crypto ban since the start of the year, cracking down on individuals and projects alike.6632 Total v
Biggest Movers: ETC Remains Near 2-Month Low, LTC Down by 4% on Monday
Biggest Movers: ETC Remains Near 2-Month Low, LTC Down by 4% on Monday Ethereum classic remained near a two-month low on March 6, as a cloud of uncertainty continued to hover over
Data Shows Global Financial Conditions Tightest in 2 Years, Shaky Bond Markets Point to Long-Run Inflation
Data Shows Global Financial Conditions Tightest in 2 Years, Shaky Bond Markets Point to Long-Run Inflation At the end of the trading day on Monday, Wall Street was roiled once aga
Alleged Kenyan Bill Proposes Widening Definition of Securities to Include Crypto Assets
Alleged Kenyan Bill Proposes Widening Definition of Securities to Include Crypto Assets A bill seeking to put blockchain and crypto assets under the purview of the Kenyan Capital M
Marathon Purchases 10,000 Bitcoin Miners, Machines Will Max Out 100 Megawatt Montana Facility
Marathon Purchases 10,000 Bitcoin Miners, Machines Will Max Out 100 Megawatt Montana Facility On December 9, the Nasdaq-listed cryptocurrency mining company, Mar
Oxford economist who predicted crypto going mainstream says ‘quantum economics’ is next
Tristan Greene8 hours agoOxford economist who predicted crypto going mainstream says ‘quantum economics’ is nextDavid Orrell literally wrote the book on Quantum Economics.2013 Total views8 Total sharesListen to artic
EU committees approve ban on anonymous crypto transactions via hosted wallets
Amaka Nwaokocha10 hours agoEU committees approve ban on anonymous crypto transactions via hosted walletsThe recent Anti-Money Laundering legislation imposes certain limits for cash transactions and anonymous cryptocurren
Nigerian Banks Resume Dispensing Recently Demonetized Naira Banknotes
Nigerian Banks Resume Dispensing Recently Demonetized Naira Banknotes According to local reports, Nigerian financial institutions have begun abiding by a Supreme Court ruling that
Halving 2024: Where is Bitcoin heading next?
Marco Castrovilli5 hours agoHalving 2024: Where is Bitcoin heading next?In our latest Cointelegraph video, we explain everything you need to know about the imminent Bitcoin halving, assessing its impact on both the minin