Tracking Down and Fixing a Zsh History Data Loss Bug

Main text:

Among Zsh users, reports of previously executed commands disappearing from the history file are occasionally seen. In a report on Lobsters from michael.stapelberg.ch by stapelberg, the author details the process of independently tracking down and ultimately identifying the cause of this problem, which he had experienced for years. The lost history spanned entries covering years, and the file itself showed no visible corruption.

Conditions for Reproducing the Symptom

The author uses Zsh on a daily basis and would sometimes fail to find a recent command with the Ctrl+R history search. Specifically, commands executed the previous day were not in the history, and the .zsh_history file only contained old entries from years ago. The line count of the file varied each time, and no invalid characters or incomplete lines were found. Every time the problem occurred, the history was restored from the daily backup, but the root cause had not been identified.

The Author’s Zsh History Settings

The author’s Zsh history settings may have been involved in the background of the problem. The .zshrc contained the following options.

HISTSIZE=4000
HISTFILE=~/.zsh_history
SAVEHIST=10000000
setopt HIST_IGNORE_DUPS
setopt INC_APPEND_HISTORY
unsetopt SHARE_HISTORY

In this configuration, HISTSIZE is limited to 4000 lines, while SAVEHIST is set to 10 million lines. INC_APPEND_HISTORY causes history to be appended immediately when a command is executed, and SHARE_HISTORY is disabled, so multiple shell sessions do not share each other’s history but instead write to the same file. In effect, an environment was created in which multiple Zsh processes independently access ~/.zsh_history.

Limitations of inotify Monitoring

After seeking advice on Mastodon in December 2024, the use of a filesystem change monitoring mechanism such as inotify was suggested. When monitoring .zsh_history with inotifywait on Linux, it was observed that multiple Zsh processes frequently performed operations such as OPEN, ACCESS, and CLOSE_WRITE. However, because all shells access the same file, it was not possible to simply identify which process was performing the truncation operation. fsevents and other monitoring methods were also considered, but faced the same constraints.

Identifying the Cause via Core Dump Analysis

Since the conventional monitoring approach failed, the author adopted a method of patching Zsh to intentionally crash it. Analyzing the core dump generated at crash time made it possible to understand in detail the internal state related to history management. With this method, he determined that the problem was caused by race conditions among multiple Zsh processes. Specifically, it became clear that during INC_APPEND_HISTORY operation, when different processes simultaneously append to the end of the file, there is a flaw in file pointer management.

Fix in Zsh 5.9.2

The findings were reported to the Zsh development team, and a fix was incorporated. Zsh 5.9.2 (released on July 12, 2026) includes a fix for this bug. The upstream fix number is 53454, and the Lobsters article recommends verifying the behavior after the fix. The author points out that the reason this problem remained unresolved for years was the complexity of the reproduction conditions and the difficulty of monitoring.

Technical Details and Scope of Impact

Zsh’s history management is designed with an emphasis on sharing state between shell sessions. Options such as HIST_IGNORE_DUPS and INC_APPEND_HISTORY improve usability, but they also complicate the maintenance of consistency during concurrent access. Because this bug manifests only under specific configuration conditions, it does not affect all users. However, in large-scale development environments or cases where history is retained over long periods, the risk of data loss increases.

A similar problem can be seen in SQLite WAL mode blind spot: lock contention even when read-only. File access by multiple processes requires careful design. Even in shells other than Zsh, similar vulnerabilities may lurk depending on the history management implementation.

Future Challenges

Although the direct bug was resolved with the fix in Zsh 5.9.2, the need for comprehensive testing to guarantee the integrity of history files has been highlighted. Long-term bug tracking in open source projects reflects the gap between users’ real environments and development environments. This investigation showed that core dump analysis, a classic debugging method, remains effective even in modern complex software.

Editorial Opinion

In the short term, upgrading to Zsh 5.9.2 is recommended. This is expected to reduce the risk of history loss and improve developer efficiency. On the other hand, in environments that continue to use older versions, regular backups and monitoring of history files will be essential. This fix not only improves shell stability, but also serves as a reminder of similar concurrent access bugs. From a long-term perspective, there is a possibility that the history management mechanisms of shells including Zsh will be reevaluated. Sharing state across multiple sessions enhances convenience, but mechanisms for maintaining consistency need to be strengthened. The use of monitoring APIs such as Linux kernel inotify and fsevents will also likely be reconsidered as a debugging technique. Going forward, improvements to history file format and access control are expected. As a question from the editorial department, access contention for shared files by multiple processes is not limited to Zsh. Even long-running editors such as Vim and Emacs can experience similar problems with configuration files or swap files between sessions.

References

• “Tracking down a Zsh history data loss bug”, by michael.stapelberg.ch by stapelberg — Lobsters, 2026-08-09T08:16:46.000Z (ARR)
• Source URL: https://michael.stapelberg.ch/posts/2026-08-09-zsh-history-truncation-bug/