Abstract
We introduce a unifying framework that brings two leading "locate-and-edit"model editing techniques -- ROME and MEMIT -- under a single conceptualumbrella, optimizing for the same goal, which we call thepreservation-memorization objective. ROME uses an equality constraint toperform one edit at a time, whereas MEMIT employs a more flexible least-squareconstraint that allows for batched edits. Following thepreservation-memorization objective, we present Equality-constrained Mass ModelEditing algorithm for Transformers or EMMET, a new batched memory-editingalgorithm that uses a closed-form solution for the equality-constrained versionof the preservation-memorization objective. EMMET is a batched-version of ROMEand is able to perform batched-edits up to a batch-size of 10,000 with verysimilar performance to MEMIT across multiple dimensions. With EMMET, we unifyand achieve symmetry within the "locate-and-edit" algorithms, allowingbatched-editing using both objectives.