Mutsu City was founded in 1 September 1959 when Tanabu-machi, known for its connection with the Tonami clan (formerly Nanbu clan), and Ominato-machi, where the former Navy base used to be, merged and was named Ominato-Tanabu City, the 8th city in Aomori Prefecture. Before the  merger, the two towns had been designated as a village in 1889 as part of the organization of municipalities after merging with neighboring villages (Tanabu village merged with Okunai and Nakanosawa villages, Ominato village merged with Taihei and Jogasawa villages). After this, Tanabu became a town in 1899 and Ominato in 1928.
 
In 1953, the law for promoting mergers between municipalities was implemented. In July 1954, Aomori Prefecture held a conference to explain about the first draft of the plan for merging Ominato and
Tanabu towns at the Ominato Hotel, putting a start to making the draft more applicable. Because people of both towns had mixed feelings and their characters differed from one another, it was not easy to complete the merger. But after five years from the explanation on 1 September 1959, the two towns merged and became Ominato-Tanabu City.
 
On 1 August 1960, the city changed its name to Mutsu City and the official name was written only in hiragana letters.
 
In 2005, when there was a boom in mergers between municipalities all over the country, the four municipalities of Mutsu City, Sendai-machi, Ohata-machi, and Wakinosawa-mura merged to become the new Mutsu City on 14 March.
 
The city was named Mutsu because the entire northern region of Tohoku had always been called "Mutsu-no-kuni" (The land of Mutsu), and because it is at the center of the Shimokita Peninsula that surrounds Mutsu Bay, and also because the name in Kanji was pronounced "Mutsu." This is why the city's name is written in hiragana instead of Kanji.