Explainable and Efficient Editing for Large Language Models

Abstract

Large Language Models (LLMs) exhibit remarkable capabilities in storing and retrieving vast amounts of factual knowledge. However, they retain outdated or incorrect information from Web corpora. Since full retraining is costly, locate-and-edit model editing methods offer a feasible alternative. Current methods typically follow a two-stage paradigm: (1) identifying critical layers that store knowledge and (2) updating their parameters to store new knowledge. However, both phases have their inherent limitations. Firstly, layer identification is independent of the knowledge being updated, ignoring the differences in knowledge storage patterns. Secondly, parameter updating suffers from high computational overhead due to gradient descent. To solve these, we propose an Explainable and effiCient model Editing method, termed ECE. Specifically, we integrate LLM explainability into the editing process, enabling the adaptive identification of the crucial neurons. Through clustering similar knowledge, we enable batch optimization in a single gradient step, significantly reducing computational time without compromising effectiveness. Extensive experiments demonstrate that ECE can achieve superior performance, showcasing the potential of explainability-driven editing methods for LLMs. Code is available at https://github.com/tianyuzhangterry/ECE.

@inproceedings{DBLP:conf/www/ZhangFJBW025,
  author       = {Tianyu Zhang and
                  Junfeng Fang and
                  Houcheng Jiang and
                  Baolong Bi and
                  Xiang Wang and
                  Xiangnan He},
  editor       = {Guodong Long and
                  Michale Blumestein and
                  Yi Chang and
                  Liane Lewin{-}Eytan and
                  Zi Helen Huang and
                  Elad Yom{-}Tov},
  title        = {Explainable and Efficient Editing for Large Language Models},
  booktitle    = {Proceedings of the {ACM} on Web Conference 2025, {WWW} 2025, Sydney,
                  NSW, Australia, 28 April 2025- 2 May 2025},
  pages        = {1963--1976},
  publisher    = {{ACM}},
  year         = {2025},
  url          = {https://doi.org/10.1145/3696410.3714835},
  doi          = {10.1145/3696410.3714835},
  timestamp    = {Sun, 02 Nov 2025 21:27:17 +0100},
  biburl       = {https://dblp.org/rec/conf/www/ZhangFJBW025.bib},
  bibsource    = {dblp computer science bibliography, https://dblp.org}
}

Tianyu Zhang

张天宇

Houcheng Jiang

姜厚丞

Xiang Wang

王翔 教授

Xiangnan He

何向南 教授