Fun

News Feed - 2023-10-03 02:10:00

Tristan Greene6 hours agoResearchers find LLMs like ChatGPT output sensitive data even after it’s been ‘deleted’According to the scientists, there’s no universal method by which data can be deleted from a pretrained large language model.2784 Total views12 Total sharesListen to article 0:00NewsJoin us on social networksA trio of scientists from the University of North Carolina, Chapel Hill recently published preprint artificial intelligence (AI) research showcasing how difficult it is to remove sensitive data from large language models (LLMs) such as OpenAI’s ChatGPT and Google’s Bard. 


According to the researchers" paper, the task of “deleting” information from LLMs is possible, but it’s just as difficult to verify the information has been removed as it is to actually remove it.


The reason for this has to do with how LLMs are engineered and trained. The models are pretrained on databases and then fine-tuned to generate coherent outputs (GPT stands for “generative pretrained transformer”).


Once a model is trained, its creators cannot, for example, go back into the database and delete specific files in order to prohibit the model from outputting related results. Essentially, all the information a model is trained on exists somewhere inside its weights and parameters where they’re undefinable without actually generating outputs. This is the “black box” of AI.


A problem arises when LLMs trained on massive datasets output sensitive information such as personally identifiable information, financial records, or other potentially harmful and unwanted outputs.


Related:Microsoft to form nuclear power team to support AI: Report


In a hypothetical situation where an LLM was trained on sensitive banking information, for example, there’s typically no way for the AI’s creator to find those files and delete them. Instead, AI devs use guardrails such as hard-coded prompts that inhibit specific behaviors or reinforcement learning from human feedback (RLHF).


In an RLHF paradigm, human assessors engage models with the purpose of eliciting both wanted and unwanted behaviors. When the models’ outputs are desirable, they receive feedback that tunes the model toward that behavior. And when outputs demonstrate unwanted behavior, they receive feedback designed to limit such behavior in future outputs.Despite being “deleted” from a model"s weights, the word “Spain” can still be conjured using reworded prompts. Image source: Patil, et. al., 2023


However, as the UNC researchers point out, this method relies on humans finding all the flaws a model might exhibit, and even when successful, it still doesn’t “delete” the information from the model.


Per the team’s research paper:“A possibly deeper shortcoming of RLHF is that a model may still know the sensitive information. While there is much debate about what models truly ‘know’ it seems problematic for a model to, e.g., be able to describe how to make a bioweapon but merely refrain from answering questions about how to do this.”


Ultimately, the UNC researchers concluded that even state-of-the-art model editing methods, such as Rank-One Model Editing “fail to fully delete factual information from LLMs, as facts can still be extracted 38% of the time by whitebox attacks and 29% of the time by blackbox attacks.”


The model the team used to conduct their research is called GPT-J. While GPT-3.5, one of the base models that power ChatGPT, was fine-tuned with 170 billion parameters, GPT-J only has 6 billion.


Ostensibly, this means the problem of finding and eliminating unwanted data in an LLM such as GPT-3.5 is exponentially more difficult than doing so in a smaller model.


The researchers were able to develop new defense methods to protect LLMs from some “extraction attacks” — purposeful attempts by bad actors to use prompting to circumvent a model’s guardrails in order to make it output sensitive information


However, as the researchers write, “the problem of deleting sensitive information may be one where defense methods are always playing catch-up to new attack methods.”# AI# Machine Learning# ChatGPTAdd reactionAdd reactionRead moreHow to use index funds and ETFs for passive crypto incomeAI tech boom: Is the artificial intelligence market already saturated?AI a powerful tool for devs to change gaming, says former Google gaming head

News Feed

Indian Government Prepares to Fast Track Crypto Bill — Plans to Introduce Cryptocurrency Law in a Month: Report
Indian Government Prepares to Fast Track Crypto Bill — Plans to Introduce Cryptocurrency Law in a Month: Report The Indian government is reportedly considering
Hackers Are Using Three Fake Crypto-Related Apps to Drain Wallets
Hackers Are Using Three Fake Crypto-Related Apps to Drain Wallets While bitcoin (BTC) prices are increasing, crypto-related scams appear on the scene to take adv
Next Crypto to 1,000x as Bitcoin Targets $135K by June and Trump Predicts Market Explosion
Reason to trust Strict editorial policy that focuses on accuracy, relevance, and impartiality Created by industry experts and meticulously reviewed The highest standards in reporting and pu
‘No Plan to Issue CBDC’ — Bank of Japan Governor
"No Plan to Issue CBDC" — Bank of Japan Governor Haruhiko Kuroda, the governor of the Bank of Japan (BOJ), has said the bank is currently not planning to launch a digital currenc
US Lawmaker Calls on SEC to Issue Crypto Regulations — Says ‘a Formal Regulatory Process Is Needed Now’
US Lawmaker Calls on SEC to Issue Crypto Regulations — Says "a Formal Regulatory Process Is Needed Now" A U.S. senator has called on the Securities and Exchange Commission (SEC)
Crypto Markets Suffer Heavy Losses, Bitcoin Price Sinks More Than 25% in 24 Hours
Crypto Markets Suffer Heavy Losses, Bitcoin Price Sinks More Than 25% in 24 Hours Cryptocurrency markets have suffered significant losses during the last 24 hour
Martin Young3 hours agoEtherHiding: Hackers create novel way to hide malicious code in blockchainsThreat actors have worked out a way to hide malicious payloads in Binance smart contracts to lure victims into updating th
PS4 Hits 30 Million US Units Sold, Makes Sony Next Gen’s Console-to-Beat
Sony"s PlayStation 4 console has all but won this generation of console wars, selling 30 million units in the United States alone. | Source: AFP PHOTO / JOEL SAWSony’s PlaySta
P2P Trading Resumes on Okex While Withdrawals Freeze Still in Effect: Tron Foundation Announces Cashout Plan for TRX Holders
P2P Trading Resumes on Okex While Withdrawals Freeze Still in Effect: Tron Foundation Announces Cashout Plan for TRX Holders Asian cryptocurrency exchange Okex s
Martin Young3 hours agoNorth Korean Lazarus Group amasses over $40M in Bitcoin, data revealsThe North Korean hacking collective has at least $47 million in cryptocurrency, including Bitcoin, Ether, BNB and various stable
Dogecoin Whales Buy Over 80 Million DOGE In 24 Hours – Sign Of Recovery​?
Reason to trust Strict editorial policy that focuses on accuracy, relevance, and impartiality Created by industry experts and meticulously reviewed The highest standards in reporting and pu
Meme Token Economy Climbs 14% in 2 Weeks — DOGE, SHIB Command 83% Dominance
Meme Token Economy Climbs 14% in 2 Weeks — DOGE, SHIB Command 83% Dominance Toward the end of this week, crypto-assets rebounded in value after meandering for two weeks and conso