• About
  • Advertise
  • Privacy & Policy
  • Contact
HK Businesswire
  • Home
  • News
    • All
    • Business
    • Politics
    • PR Newswire
    • Science
    • World

    EC election ‘open, honest and fair’, says David Lok

    Chongqing International Talent Exchange Conference Opens, Drawing Nobel Laureates and Global Innovators

    Chongqing International Talent Exchange Conference Opens, Drawing Nobel Laureates and Global Innovators

    APEC Business Travel Card is Linking Zhongshan with Asia-Pacific

    APEC Business Travel Card is Linking Zhongshan with Asia-Pacific

    Fossil fuels still missing from latest COP31 draft

    HK-Chile working-holiday scheme to start next year

    HK to contribute to global bay area cooperation: FS

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • PR Newswire
  • Business
  • World
  • Entertainment
  • Sports
  • Tech
    • All
    • Apps
    • Gadget
    • Mobile
    • Startup

    Cathay Cargo Launches AI Compliance Tool to Streamline Global Trade Operations

    PCCW Global and Druid Technology Partner on Belt and Road IoT Connectivity

    Alipay Launches Domestic Full-Stack Intelligent Agent Platform in Hangzhou

    HKT Posts 4% Profit Rise Amid Steady Local Recovery and AI-Driven Demand

    Alipay Launches AI-Powered Version ‘Abao’ to Streamline Services

    Xiaohongshu Prepares Confidential Hong Kong IPO Filing

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
  • Feature
No Result
View All Result
  • Home
  • News
    • All
    • Business
    • Politics
    • PR Newswire
    • Science
    • World

    EC election ‘open, honest and fair’, says David Lok

    Chongqing International Talent Exchange Conference Opens, Drawing Nobel Laureates and Global Innovators

    Chongqing International Talent Exchange Conference Opens, Drawing Nobel Laureates and Global Innovators

    APEC Business Travel Card is Linking Zhongshan with Asia-Pacific

    APEC Business Travel Card is Linking Zhongshan with Asia-Pacific

    Fossil fuels still missing from latest COP31 draft

    HK-Chile working-holiday scheme to start next year

    HK to contribute to global bay area cooperation: FS

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • PR Newswire
  • Business
  • World
  • Entertainment
  • Sports
  • Tech
    • All
    • Apps
    • Gadget
    • Mobile
    • Startup

    Cathay Cargo Launches AI Compliance Tool to Streamline Global Trade Operations

    PCCW Global and Druid Technology Partner on Belt and Road IoT Connectivity

    Alipay Launches Domestic Full-Stack Intelligent Agent Platform in Hangzhou

    HKT Posts 4% Profit Rise Amid Steady Local Recovery and AI-Driven Demand

    Alipay Launches AI-Powered Version ‘Abao’ to Streamline Services

    Xiaohongshu Prepares Confidential Hong Kong IPO Filing

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
  • Feature
No Result
View All Result
HK Businesswire
No Result
View All Result
Home News Science

Researchers glimpse the inner workings of protein language models

David Lee by David Lee
18 August 2025
in Science
0
Researchers glimpse the inner workings of protein language models
0
SHARES
3
VIEWS
Share on FacebookShare on Twitter

Within the past few years, models that can predict the structure or function of proteins have been widely used for a variety of biological applications, such as identifying drug targets and designing new therapeutic antibodies.These models, which are based on large language models (LLMs), can make very accurate predictions of a protein’s suitability for a given application. However, there’s no way to determine how these models make their predictions or which protein features play the most important role in those decisions.In a new study, MIT researchers have used a novel technique to open up that “black box” and allow them to determine what features a protein language model takes into account when making predictions. Understanding what is happening inside that black box could help researchers to choose better models for a particular task, helping to streamline the process of identifying new drugs or vaccine targets.“Our work has broad implications for enhanced explainability in downstream tasks that rely on these representations,” says Bonnie Berger, the Simons Professor of Mathematics, head of the Computation and Biology group in MIT’s Computer Science and Artificial Intelligence Laboratory, and the senior author of the study. “Additionally, identifying features that protein language models track has the potential to reveal novel biological insights from these representations.”Onkar Gujral, an MIT graduate student, is the lead author of the study, which appears this week in the Proceedings of the National Academy of Sciences. Mihir Bafna, an MIT graduate student, and Eric Alm, an MIT professor of biological engineering, are also authors of the paper.Opening the black boxIn 2018, Berger and former MIT graduate student Tristan Bepler PhD ’20 introduced the first protein language model. Their model, like subsequent protein models that accelerated the development of AlphaFold, such as ESM2 and OmegaFold, was based on LLMs. These models, which include ChatGPT, can analyze huge amounts of text and figure out which words are most likely to appear together.Protein language models use a similar approach, but instead of analyzing words, they analyze amino acid sequences. Researchers have used these models to predict the structure and function of proteins, and for applications such as identifying proteins that might bind to particular drugs.In a 2021 study, Berger and colleagues used a protein language model to predict which sections of viral surface proteins are less likely to mutate in a way that enables viral escape. This allowed them to identify possible targets for vaccines against influenza, HIV, and SARS-CoV-2.However, in all of these studies, it has been impossible to know how the models were making their predictions.“We would get out some prediction at the end, but we had absolutely no idea what was happening in the individual components of this black box,” Berger says.In the new study, the researchers wanted to dig into how protein language models make their predictions. Just like LLMs, protein language models encode information as representations that consist of a pattern of activation of different “nodes” within a neural network. These nodes are analogous to the networks of neurons that store memories and other information within the brain.The inner workings of LLMs are not easy to interpret, but within the past couple of years, researchers have begun using a type of algorithm known as a sparse autoencoder to help shed some light on how those models make their predictions. The new study from Berger’s lab is the first to use this algorithm on protein language models.Sparse autoencoders work by adjusting how a protein is represented within a neural network. Typically, a given protein will be represented by a pattern of activation of a constrained number of neurons, for example, 480. A sparse autoencoder will expand that representation into a much larger number of nodes, say 20,000.When information about a protein is encoded by only 480 neurons, each node lights up for multiple features, making it very difficult to know what features each node is encoding. However, when the neural network is expanded to 20,000 nodes, this extra space along with a sparsity constraint gives the information room to “spread out.” Now, a feature of the protein that was previously encoded by multiple nodes can occupy a single node.“In a sparse representation, the neurons lighting up are doing so in a more meaningful manner,” Gujral says. “Before the sparse representations are created, the networks pack information so tightly together that it’s hard to interpret the neurons.”Interpretable modelsOnce the researchers obtained sparse representations of many proteins, they used an AI assistant called Claude (related to the popular Anthropic chatbot of the same name), to analyze the representations. In this case, they asked Claude to compare the sparse representations with the known features of each protein, such as molecular function, protein family, or location within a cell.By analyzing thousands of representations, Claude can determine which nodes correspond to specific protein features, then describe them in plain English. For example, the algorithm might say, “This neuron appears to be detecting proteins involved in transmembrane transport of ions or amino acids, particularly those located in the plasma membrane.”This process makes the nodes far more “interpretable,” meaning the researchers can tell what each node is encoding. They found that the features most likely to be encoded by these nodes were protein family and certain functions, including several different metabolic and biosynthetic processes.“When you train a sparse autoencoder, you aren’t training it to be interpretable, but it turns out that by incentivizing the representation to be really sparse, that ends up resulting in interpretability,” Gujral says.Understanding what features a particular protein model is encoding could help researchers choose the right model for a particular task, or tweak the type of input they give the model, to generate the best results. Additionally, analyzing the features that a model encodes could one day help biologists to learn more about the proteins that they are studying.“At some point when the models get a lot more powerful, you could learn more biology than you already know, from opening up the models,” Gujral says.The research was funded by the National Institutes of Health. 

Tags: Science
David Lee

David Lee

Read More

New approach enables physically consistent AI models of chaotic systems

20 November 2026

Q&A: Enabling more human-like interactions and understanding in multimodal foundation models

20 November 2026
  • Trending
  • Comments
  • Latest
Newgen reports Revenue of ₹357 crores in Q1 FY’27, up 11% Q1 YoY; Profit After Tax at ₹63 crores, up 26% Q1 YoY

Newgen Software and the Asian Institute of Digital Finance Forge Strategic Partnership to Shape the Future of Enterprise AI

10 November 2026

World Winter Sports Expo 2026 Opens, Charting a New Decade for Ice and Snow and Outdoor Industries

17 October 2026

HK five-year plan offers UK long-term prospects: CE

10 November 2026
Patriot Launches AI Matrix SSD and DRAM Series Engineered for Local AI Workloads

Patriot Launches AI Matrix SSD and DRAM Series Engineered for Local AI Workloads

19 November 2026

EC election ‘open, honest and fair’, says David Lok

22 November 2026
Chongqing International Talent Exchange Conference Opens, Drawing Nobel Laureates and Global Innovators

Chongqing International Talent Exchange Conference Opens, Drawing Nobel Laureates and Global Innovators

22 November 2026
APEC Business Travel Card is Linking Zhongshan with Asia-Pacific

APEC Business Travel Card is Linking Zhongshan with Asia-Pacific

22 November 2026

Fossil fuels still missing from latest COP31 draft

22 November 2026

Recent News

EC election ‘open, honest and fair’, says David Lok

22 November 2026
Chongqing International Talent Exchange Conference Opens, Drawing Nobel Laureates and Global Innovators

Chongqing International Talent Exchange Conference Opens, Drawing Nobel Laureates and Global Innovators

22 November 2026
APEC Business Travel Card is Linking Zhongshan with Asia-Pacific

APEC Business Travel Card is Linking Zhongshan with Asia-Pacific

22 November 2026

Fossil fuels still missing from latest COP31 draft

22 November 2026
HK Businesswire

Stay ahead with the latest insights on Hong Kong’s economy, finance, and investments. From market trends to policy updates, we bring you in-depth analysis and expert opinions.

📩 Subscribe to our newsletter for exclusive updates.
📍 Follow us on social media for real-time news.
📧 Contact us: info@hongkong-invest.com

Follow Us

  • About
  • Advertise
  • Privacy & Policy
  • Contact

© 2025 by HKBusinesswire.com

No Result
View All Result

© 2025 by HKBusinesswire.com